For scientific machine learning tasks with a lot of custom code, picking the right Automatic Differentiation (AD) system matters. Our Julia package DifferentiationInterface$.$jl provides a common frontend to a dozen AD backends, unlocking easy comparison and modular development. In particular, its built-in preparation mechanism leverages the strengths of each backend by amortizing one-time computations. This is key to enabling sophisticated features like sparsity handling without putting additional burdens on the user.

Abstract: Scientific machine learning (SciML) is the burgeoning field combining scientific knowledge with machine learning for data-efficient predictive modeling. We will introduce SciML as the key to effective learning in many engineering applications, such as improving the fidelity of climate models to accelerating clinical trials. This will lead us to the question on the frontier of SciML: what about stiffness? Stiffness is a pervasive quality throughout engineering systems and the most common cause of numerical difficulties in simulation. We will see that handling stiffness in learning, and thus real-world models, requires new training techniques.